Covariance-aware Feature Alignment with Pre-computed Source Statistics for Test-time Adaptation to Multiple Image Corruptions. (arXiv:2204.13263v2 [cs.LG] UPDATED)
Real-world image recognition systems often face corrupted input images, which
cause distribution shifts and degrade the performance of models. These systems
often use a single prediction model in a central server and process images sent
from various environments, such as cameras distributed in cities or cars. Such
single models face images corrupted in heterogeneous ways in test time. Thus,
they require to instantly adapt to the multiple corruptions during testing
rather than being re-trained at a high cost. Test-time adaptation (TTA), which
aims to adapt models without accessing the training dataset, is one of the
settings that can address this problem. Existing TTA methods indeed work well
on a single corruption. However, the adaptation ability is limited when
multiple types of corruption occur, which is more realistic. We hypothesize
this is because the distribution shift is more complicated, and the adaptation
becomes more difficult in case of multiple corruptions. In fact, we
experimentally found that a larger distribution gap remains after TTA. To
address the distribution gap during testing, we propose a novel TTA method
named Covariance-Aware Feature alignment (CAFe). We empirically show that CAFe
outperforms prior TTA methods on image corruptions, including multiple types of
corruptions.